Method for estimating lane information, and electronic device

ABSTRACT

Provided is an Artificial Intelligence (AI) system for simulating a human brain&#39;s functions, such as recognition, decision, etc., by using a machine learning algorithm such as deep learning, etc. and applications of the AI system. Provided is an electronic device including: a camera configured to capture an outside image of a vehicle, and a processor configured to execute one or more instructions stored in a memory, wherein the processor executes the one or more instructions to: determine, from the captured image, at least one object for estimating lane information; estimate, from the image, lane information of a road on which the vehicle is traveling, based on a distance between the determined at least one object and the vehicle and a vanishing point of the image; and output guide information for guiding driving of the vehicle based on the estimated lane information.

TECHNICAL FIELD

Various embodiments relate to a method and electronic device forestimating lane information, and more particularly, to a method andelectronic device for estimating and outputting lane information even inbad weather.

BACKGROUND ART

An artificial intelligence (AI) system is a computer system forimplementing human-level intelligence. Unlike existing rule-based smartsystems, the AI system is a system in which machines learn, judge, andbecome smarter. The more the AI system is used, the higher therecognition rate of the AI system becomes so that the AI system canunderstand a user's tastes more accurately. For this reason, typicalrule-based smart systems are being gradually replaced by deeplearning-based AI systems.

AI technology consists of machine learning (deep learning) and elementtechnology based on the machine learning.

The machine learning is algorithm technology that itselfclassifies/learns the characteristics of input data. The elementtechnology uses a machine learning algorithm such as deep learning toimitate a human brain's functions such as recognition and determination.The machine learning is composed of technical fields includinglinguistic comprehension, visual comprehension, inference/prediction,knowledge representation, motion control, etc.

Various applications of the AI technology are as follows. The linguisticcomprehension is technology for recognizing and applying/processinghuman language/characters, and includes natural language processing,machine translation, a dialogue system, query response, voicerecognition/synthesis, etc. The visual comprehension is technology forrecognizing/processing objects as done in human vision, and includesobject recognition, object tracking, image search, human recognition,scene understanding, spatial understanding, and image enhancement. Theinference/prediction is technology for judging and logically inferringand predicting information, and includes knowledge/probability-basedinference, optimization prediction, preference-based planning,recommendation, etc. The knowledge representation is technology forautomatically processing human experience information with knowledgedata, and includes knowledge construction (datacreation/classification), knowledge management (use of data), etc. Themotion control is technology for controlling autonomous driving ofvehicles, motions of robots, etc., and includes motion control(navigation, collision avoidance, driving), operating control (behaviorcontrol), etc.

Along with development of technologies that are applied to vehicles,various methods for an autonomous driving system (ADS) or an advanceddriver assistance system (ADAS) are being developed.

Meanwhile, according to weather of a region where a vehicle is operatedor a time at which the vehicle is operated, it is often the case that adriver cannot acquire sufficient visual information required fordriving. Accordingly, there is demand for technology for providinginformation about a lane on which a vehicle is located to assist adriver with driving operations or for an ADS to safely control thevehicle in an environment where obstructions exist.

DESCRIPTION OF EMBODIMENTS Technical Problem

Various embodiments are to provide a method and device for estimatinglane information of a road, on which a vehicle is traveling, in anenvironment including a visibility obstruction such as bad weather.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a schematic diagram showing an example of estimating laneinformation by an electronic device according to an embodiment.

FIG. 2 is a flowchart showing an operation method of an electronicdevice according to an embodiment.

FIG. 3 is a diagram for describing a method of, performed by anelectronic device, converting images, according to an embodiment.

FIG. 4 is a diagram for describing a method of, performed by anelectronic device, estimating lane information of a road, according toan embodiment.

FIG. 5 is a diagram for describing a method of, performed by anelectronic device, estimating lane information of a road, according toan embodiment.

FIG. 6 is a diagram for describing a method of, performed by anelectronic device, outputting guide information, according to anembodiment.

FIGS. 7 and 8 are block diagrams showing a configuration of anelectronic device according to an embodiment.

FIG. 9 is a block diagram of a processor according to an embodiment.

FIG. 10 is a block diagram of a data learner according to an embodiment.

FIG. 11 is a block diagram of a data recognizer according to anembodiment.

FIG. 12 is a diagram for describing an example in which an electronicdevice according to an embodiment interworks with a server to learn andrecognize data.

BEST MODE

An electronic device according to an embodiment includes: a cameraconfigured to capture an outside image of a vehicle; a memory storingone or more instructions; and a processor configured to execute the oneor more instructions stored in the memory, wherein the processorexecutes the one or more instructions to: determine, from the capturedimage, at least one object for estimating lane information; estimate,from the image, lane information of a road on which the vehicle istraveling, based on a distance between the determined at least oneobject and the vehicle and a vanishing point of the image; and outputguide information for guiding driving of the vehicle based on theestimated lane information.

An operation method of an electronic device according to an embodimentincludes: acquiring an outside image of the vehicle; determining, fromthe acquired image, at least one object for estimating lane information;estimating, from the image, lane information of a road on which thevehicle is traveling, based on a distance between the determined atleast one object and the vehicle and a vanishing point of the image; andoutputting guide information for guiding driving of the vehicle based onthe estimated lane information.

A computer-readable recording medium storing a program for executing theabove-described method on a computer.

Mode of Disclosure

Hereinafter, embodiments of the disclosure will be described in detailwith reference to the accompanying drawings so that the disclosure maybe readily implemented by those skilled in the art. However, thedisclosure is not restricted by these embodiments but can be implementedin many different forms. Also, in the drawings, parts irrelevant to thedescription are omitted for the simplicity of explanation, and likereference numerals refer to like elements throughout the specification.

It will be understood that when a component is referred to as being“connected” to another component, it can be “directly connected” or“electrically connected” to the other component with an interveningcomponent. Also, it will be understood that when a certain part“includes” a certain component, the part does not exclude anothercomponent but can further include another component, unless the contextclearly dictates otherwise.

Hereinafter, the disclosure will be described in detail with referenceto the accompanying drawings.

<FIG. 1>

FIG. 1 is a schematic diagram showing an example of estimating laneinformation by an electronic device according to an embodiment.

Referring to FIG. 1, according to an embodiment, an electronic device100 may be installed in a vehicle 110 to estimate lane information 103of a road on which the vehicle 110 is traveling, from an outside image101 of the vehicle 110. Herein, the lane information 103 means linesthat divide the road on which the vehicle 110 is traveling into lanes(e.g., a first lane and a second lane when the road is a two-lane road).

For example, the electronic device 100 may be a mobile or fixedelectronic device that may be mounted on the vehicle 110. Also, theelectronic device 100 may include a camera for acquiring the outsideimage 101 and a display module for outputting guide information. Also,the electronic device 100 may control other devices, such as a camera, adisplay 106, a local navigation, a GPS receiver, etc., included in thevehicle 110. Also, the electronic device 100 may communicate with otherdevices included in the vehicle 110 to receive data for estimating thelane information 103 or to transmit guide information.

When a driver drives the vehicle 110 in an environment including avisibility obstruction, the driver's clear view may be not secured.Furthermore, Autonomous Driving System (ADS) or Advanced DriverAssistance System (ADAS) of the vehicle 110 may operate not properly.The electronic device 100 may estimate the lane information 103 of theroad on which the vehicle 110 is traveling, and output guide informationas shown in FIG. 1, thereby assisting safe driving of the vehicle 110.

For example, the electronic device 100 may estimate the lane information103 of the road on which the vehicle 110 is traveling by acquiring theoutside image 101 of the vehicle 110. Also, the electronic device 100may estimate the lane information 103 by using at least one objectincluded in the image 101. Herein, the object may be a subject that isincluded in the image 101 and that may be recognized by beingdistinguished from other subjects in the image 101. For example, theimage 101 may include at least one object, such as a guard rail 105 or afront vehicle 104. Also, the electronic device 100 may estimate the laneinformation 103 by using information obtained through analysis of theimage 101, such as information about the number of lanes of the road, avanishing point of the image 101, etc.

Meanwhile, according to an embodiment, the electronic device 100 may usea learning model to convert the image 101 acquired for estimating thelane information 103 such that the image 101 has visibility that isgreater than or equal to a predetermined value. Also, according to anembodiment, the learning model for converting the image 101 may be basedon learning according to Deep Neural Model (DNM) technology. Thelearning model may be an intelligence learning model.

According to an embodiment, the electronic device 100 may output guideinformation based on the estimated lane information 103. Herein, theguide information means information for guiding driving of the vehicle110. For example, the guide information may include lane information ofa road, driving speed of the vehicle 110, or danger warning informationprocessed based on the driving speed of the vehicle 110.

Also, the electronic device 100 may include the display 106 fordisplaying the guide information. According to an embodiment, thedisplay 106 may include at least one of a head-up display, a mirrordisplay, and a transparent display.

Also, the electronic device 100 may control driving of the vehicle 110based on ADS or ADAS.

According to an embodiment, the electronic device 100 may be a smartphone, a tablet PC, a smart PC, a mobile phone, personal digitalassistant (PDA), a laptop, a media player, a micro server, a globalpositioning system (GPS) device, an e-book terminal, a terminal fordigital broadcasting, a navigation, a kiosk, a MP3 player, a digitalcamera, a home appliance, or another computing device, although notlimited thereto. Also, the electronic device 100 may be a wearabledevice, such as a watch, glasses, a hair band, and a ring, including adisplay function and a data processing function, although not limitedthereto. Also, the electronic device 100 may include all kinds ofdevices that may process data and provide the processed data.

Meanwhile, in FIG. 1, the electronic device 100 acquires a front-viewimage of the vehicle 110 traveling, and estimates the lane information103 of lanes ahead of the vehicle 110 by using the acquired image.However, the disclosure is not limited to the example of FIG. 1. Forexample, the electronic device 100 may acquire a rear-view image of thevehicle 110 traveling, and estimate the lane information 103 by usingthe acquired image. In this case, the electronic device 100 may providerear lane information of the road on which the vehicle 110 is travelingto assist the driver's driving.

In FIG. 1, the electronic device 100 is shown to be separated from thevehicle 110, although not limited thereto. The electronic device 100 maybe integrated into the vehicle 110 to be implemented as a componentincluded in the vehicle 110. For example, the electronic device 100 maybe implemented as a processor included in the vehicle 110. For example,the processor may include a micro controller unit (MCU) included in thevehicle 110. Also, the vehicle 110 may include a communication modulethat may communicate with a memory or an external device storing thelane information 103 and data required for operating the processor.

<FIG. 2>

FIG. 2 is a flowchart showing an operation method of an electronicdevice according to an embodiment.

Referring to FIG. 2, in operation 202, the electronic device 100 mayacquire an outside image of a vehicle. Herein, the outside image of thevehicle may be an image of a space that may be sensed by a camera or asensor in an outside space of the vehicle. For example, the outsideimage of the vehicle may be an image showing a front view or a rear viewof the vehicle, although not limited thereto.

According to an embodiment, the electronic device 100 may acquire anoutside image of the vehicle by using a camera. For example, the cameramay be a pinhole camera, a stereo camera, an infrared camera, or athermal image camera, although not limited thereto. For example, theelectronic device 100 may acquire the outside image of the vehicle byusing a camera included in the electronic device 100, or may receive theoutside image of the vehicle from a photographing device located outsidethe electronic device 100.

In operation 204, the electronic device 100 may determine at least oneobject for estimating lane information from the acquired image. Forexample, the electronic device 100 may analyse pixels included in theimage to extract a plurality of objects as distinguished objects. Theelectronic device 100 may extract the objects from the image by using alearning model according to deep neural network (DNN) technology.

Also, the electronic device 100 may determine at least one object forestimating lane information from among the extracted objects. Forexample, the electronic device 100 may select a predetermined objectfrom among the plurality of objects included in the image according to alane information estimation method to determine the selected object asat least one object for estimating lane information. For example, theelectronic device 100 may determine at least one among a guard rail,another vehicle traveling ahead, and another vehicle traveling behind,included in the image, as at least one object for estimating laneinformation.

According to an embodiment, the electronic device 100 may use a learningmodel according to the DNN technology to determine at least one objectfor estimating lane information from the acquired image. For example,the electronic device 100 may improve visibility of the image by usingthe learning model. Also, the electronic device 100 may determine atleast one object for estimating lane information from the image whosevisibility has been improved. Use of the learning model according to theDNN technology will be described in detail with reference to FIG. 3,later.

In operation 206, the electronic device 100 may estimate laneinformation of a road on which the vehicle is traveling from the image,based on a distance between the determined object and the vehicle and avanishing point of the image.

According to an embodiment, the electronic device 100 may measure adistance between the determined object and the vehicle. For example, theelectronic device 100 may measure a distance between the determinedobject and the vehicle by using a distance measuring sensor. Or, theelectronic device 100 may measure a distance between the determinedobject and the vehicle based on experiment data stored in advance forspecific types of objects and a size of a specific object included inthe acquired image.

According to an embodiment, the electronic device 100 may predict avanishing point of the image. For example, the electronic device 100 mayextract straight lines through a lower end of a building, a guard rail,etc. among the objects included in the image to thereby predict avanishing point. More specifically, the electronic device 100 maypredict a point at which extension lines of a plurality of straightlines extracted from the image meet, as a vanishing point.

For example, the electronic device 100 may determine a road area basedon a location of the determined object, and determine a lane width ofeach lane based on the distance between the determined object and thevehicle. Also, the electronic device 100 may divide the road area by thenumber of lanes by using the vanishing point of the image as a pivot toestimate lane information of the road.

Or, for example, the electronic device 100 may estimate a lane widthbased on the distance between the determined object and the vehicle.Also, the electronic device 100 may extend straight lines having thelane width estimated by using the vanishing point of the image as thepivot to thereby estimate lane information of the road.

In operation 208, the electronic device 100 may output guide informationfor guiding driving of the vehicle, based on the estimated laneinformation. For example, the electronic device 100 may synthesize theguide information with the road area of the acquired image to displaythe synthesized result. Or, the electronic device 100 may output theguide information as sound. For example, the electronic device 100 maydetermine whether a danger occurs on a traveling path of the vehicle,based on the guide information and predetermined criterion, and outputthe determined result as sound.

<FIG. 3>

FIG. 3 is a diagram for describing a method of, performed by anelectronic device, converting images, according to an embodiment.

Referring to block 310 of FIG. 3, according to an embodiment, theelectronic device 100 may acquire a learning model based on resultsobtained by learning a relationship between a plurality of images forthe same subject. For example, the plurality of images for the samesubject may be a plurality of images obtained by photographing the sameobject at the same location and at the same angle. For example, therelationship between the plurality of images may be difference valuesbetween pixels at the same location of the images.

For example, the electronic device 100 may create the learning model bylearning a relationship between a plurality of pair images. Herein, theplurality of pair images may be a plurality of images 311 including avisibility obstruction (e.g., a low light level, a fog, yellow dust,etc.) and a plurality of images 312 including no visibility obstruction,the images 312 corresponding to the plurality of images 311.

According to another embodiment, the electronic device 100 may use afirst learning model (not shown) for image analysis created in advanceto create a second learning model 321 for image conversion. For example,the electronic device 100 may receive a first learning model for imageanalysis from another external electronic device (e.g., a serverincluding a plurality of image analysis models). Also, the electronicdevice 100 may again learn the first learning model to create the secondlearning model 321 for image conversion.

For example, the electronic device 100 may create the second learningmodel 321 by using a difference between an output image obtained bypassing an image 311 including a visibility obstruction through thefirst learning model and an image including no visibility obstruction.

Or, the electronic device 100 may receive the learned learning model 321from the outside.

Also, according to an embodiment, the electronic device 100 maydetermine at least one object for estimating lane information from aconverted image. In this case, the embodiments described above inoperation 204 of FIG. 2 may be applied.

Referring to block 320 of FIG. 3, according to an embodiment, theelectronic device 100 may use the learning model 321 acquired in block310 to convert the acquired image such that the acquired image hasvisibility that is greater than or equal to a predetermined value. Forexample, the electronic device 100 may use the learning model 321 toincrease visibility of an image including a visibility obstruction.

According to an embodiment, the learning model 321 may be a group ofalgorithms for extracting various attributes included in an image byusing results of statistical machine learning and using the attributesto identify and/or determine objects included in the image. For example,when visibility of an image is determined to be smaller than thepredetermined value, the learning model 321 may perform image conversionbased on identifications and/or determinations on objects of the image.

According to an embodiment, the learning model 321 may be an end-to-endDNN model. Herein, the end-to-end DNN model means a learning modelcapable of converting an input image into an output image withoutpost-processing.

For example, the electronic device 100 may input a first image 322including a visibility obstruction to the acquired learning model 321 toconvert the first image 322 into a second image 323. Herein, the secondimage 323 may be an image having visibility that is greater than orequal to the predetermined value.

For example, the learning model 321 may identify objects included in animage and process pixels included in each object to thereby convert theimage such that visibility of the image is greater than or equal to thepredetermined value.

According to an embodiment, the predetermined value may be determinedaccording to the learning model 321. For example, the predeterminedvalue may change when the learning model 321 is refined.

According to an embodiment, the electronic device 100 may convert animage into an image having visibility that is greater than or equal tothe predetermined value by using the learning model 321, therebyincreasing a degree of precision compared to a typical case ofestimating lane information through analysis of an image including avisibility obstruction.

<FIG. 4>

FIG. 4 is a diagram for describing a method of, performed by anelectronic device, estimating lane information of a road, according toan embodiment.

Referring to FIG. 4, according to an embodiment, the electronic device100 may estimate lane information 405, based on information obtainedthrough analysis of an acquired image and information 404 about thenumber of lanes of a road on which a vehicle is traveling. For example,the information obtained through analysis of the image may include atleast one object, such as a vanishing point of the image, a road area, aguard rail, or a front vehicle, although not limited thereto.

According to an embodiment, the electronic device 100 may determine aroad area 402 from the acquired image. For example, the electronicdevice 100 may determine the road area 402, based on a location andheight of a guard rail 401 among a plurality of objects included in theimage, acquired through analysis of the image.

According to an embodiment, the electronic device 100 may acquireinformation 404 about the number of lanes of a road on which a vehicle400 is traveling, based on position information of the vehicle 400. Forexample, the electronic device 100 may acquire the information 404 aboutthe number of lanes of the road on which the vehicle 400 is traveling,based on position information of the vehicle 400 acquired through GlobalPositioning System (GPS). For example, the electronic device 100 mayacquire the information 404 about the number of lanes of the road onwhich the vehicle 400 is traveling, from a local navigation 411 includedin the vehicle 400. Or, the electronic device 100 may acquireinformation 404 about the number of lanes stored in advance in theelectronic device 100, based on position information of the vehicle 400.

According to an embodiment, the electronic device 100 may estimate lanewidths 407 to 410 for the individual lanes of the road on which thevehicle 400 is traveling, based on a distance between the vehicle 400and a front vehicle 406 determined from the image, the determined roadarea 402, and the acquired information 404 about the number of lanes.For example, when the information 404 about the number of lanesindicates a four-lane road, the electronic device 100 may divide theroad area 402 into four lanes. Also, the electronic device 100 maydetermine a ratio of the lanes divided according to the information 404about the number of lanes with respect to the road area 402, based onthe distance between the front vehicle 406 and the vehicle 400. Morespecifically, the electronic device 100 may determine a ratio of a firstlane, a second lane, a third lane, and a fourth lane as1.1:1.2:1.2:1.0:1.1. Also, the electronic device 100 may estimate thelane widths 407 to 410 based on the determined ratio.

According to an embodiment, the electronic device 100 may estimate thelane widths 407 to 410 by referring to experiment data values stored inadvance. For example, the experiment data values stored in advance maybe information about a ratio of lane widths matching with the distancebetween the front vehicle 406 and the vehicle 400 and the information404 about the number of lanes, when the road area 402 is 1.

According to an embodiment, the electronic device 100 may estimate avanishing point of the image. Also, the embodiments described above inoperation 206 of FIG. 2 may be applied to a method of determining thedistance between the front vehicle 406 and the vehicle 400 by theelectronic device 100 and a method of determining the vanishing point403 of the image by the electronic device 100.

According to an embodiment, the electronic device 100 may estimate thelane information 405 of the road on which the vehicle 400 is traveling,based on the estimated lane widths 407 to 410 and the estimatedvanishing point 403 of the image. For example, the electronic device 100may extend straight lines of dividing the road area 402 into lanes byusing the vanishing point of the image as the pivot, based on theestimated lane widths 407 to 410, thereby estimating the laneinformation 405.

According to an embodiment, when the electronic device 100 uses theinformation 404 about the number of lanes stored in advance in thevehicle 400, the electronic device 100 may estimate the lane information405 through analysis of the acquired image without having to access adata server located outside the vehicle 400.

<FIG. 5>

FIG. 5 is a diagram for describing a method of, performed by anelectronic device, estimating lane information of a road, according toan embodiment.

Referring to FIG. 5, according to an embodiment, the electronic device100 may estimate lane information of a road based on a first frontvehicle 502 located on a traveling center line 501 of a vehicle 500.

According to an embodiment, the electronic device 100 may determine thetraveling centre line 501 of the vehicle 500. Herein, the travelingcenter line 501 may be a center line of the vehicle 500 traveling, whichmeans a center line of an image acquired through a camera.

According to an embodiment, the electronic device 100 may determine thefirst front vehicle 502 located on the travelling center line 501 as atleast one object for estimating lane information.

According to an embodiment, the electronic device 100 may estimate afirst lane width 503 of a first lane on which the vehicle 500 istraveling, based on a distance between the first front vehicle 502 andthe vehicle 500 and a vehicle width of the first front vehicle 502.Herein, the vehicle width of the first front vehicle 502 means ahorizontal size of the first front vehicle 502 detected from an acquiredimage. For example, the vehicle width may be represented in units ofpixels.

For example, the electronic device 100 may determine a distance betweenthe first front vehicle 502 and the vehicle 500 based on a size of thefirst front vehicle 502 detected from an image by using experiment datastored in advance.

For example, the electronic device 100 may estimate a sum of the vehiclewidth of the first front vehicle 502 and a first value as the first lanewidth 503 of the first lane. Also, the first value may be determinedaccording to the distance between the first front vehicle 502 and thevehicle 500. For example, the first value may be set according toexperiment data stored in advance in the electronic device 100.

Also, the electronic device 100 may estimate lane information 505 of theroad on which the vehicle 500 is traveling, based on the estimated firstlane width 503 and the vanishing point 504 of the image. For example,the electronic device 100 may extend straight lines passing thevanishing point of the image based on the first lane width 503 estimatedby using the traveling center line 501 of the image or the first frontvehicle 502, thereby estimating the lane information 505.

Meanwhile, according to an embodiment, the electronic device 100 mayestimate a line of another lane (e.g., a second lane on which a secondfront vehicle 510 is traveling) which is different from the lane onwhich the vehicle 500 is traveling, by using the similar method.

For example, the electronic device 100 may determine the second frontvehicle 510 which is not located on the traveling center line 501, as atleast one object.

Also, the electronic device 100 may estimate a second lane width of thesecond lane, based on a distance between the second front vehicle 510and the traveling center line 501, a distance between the second frontvehicle 510 and the vehicle 500, and a vehicle width of the second frontvehicle 510. For example, the electronic device 100 may estimate a sumof the vehicle width of the second front vehicle 510 and a second valueas a second lane width of the second lane. Also, the second value may bedetermined according to the distance between the second front vehicle510 and the traveling center line 501 and the distance between thesecond front vehicle 510 and the vehicle 500.

Also, the electronic device 100 may estimate lane information of thesecond lane based on the estimated second lane width and the vanishingpoint of the image. For example, the electronic device 100 may extendstraight lines passing the vanishing point of the image based on thesecond lane width estimated by using the traveling center line 501 ofthe image or the second front vehicle 510, thereby estimating laneinformation of the second lane.

Also, according to an embodiment, the electronic device 100 may outputlane information including the lane information of the lane on which thesecond front vehicle 510 is traveling and the lane information 505 ofthe lane on which the first front vehicle 502 is traveling, as guideinformation.

<FIG. 6>

FIG. 6 is a diagram for describing a method of, performed by anelectronic device, outputting guide information of a road, according toan embodiment.

Meanwhile, according to an embodiment, the electronic device 100 maycreate guide information based on estimated lane information. Forexample, the guide information may be obtained by imaging the estimatedlane information. For example, the guide information may be processed bythe electronic device 100 based on the estimated lane information anddriving information of a vehicle. For example, the electronic device 100may create the guide information based on the estimated lane informationand driving speed of the vehicle.

Referring to FIG. 6, according to an embodiment, the electronic device100 may output imaged lane information. For example, the electronicdevice 100 may display a lane 602 of a road on which the vehicle istraveling, on a display 601 (e.g., a head-up display or a transparentdisplay installed in a front portion of the vehicle) included in thevehicle, based on the estimated lane information. For example, theelectronic device 100 may display an acquired image on the display 601,and highlight the estimated lane 602 in the acquired image.

According to an embodiment, the guide information may be danger warninginformation processed by using the estimated lane information. Forexample, the electronic device 100 may predict a traveling route of thevehicle based on a driving direction, speed, etc. of the vehicle, anddetermine whether there is a danger of lane departure, based on theestimated lane information. Also, when the electronic device 100determines that there is a danger of lane departure, the electronicdevice 100 may output the corresponding guide information as sound.

Referring to FIG. 6, for example, the electronic device 100 may outputdanger warning information 603 created based on lane information. Forexample, when the electronic device 100 determines that there is adanger of lane departure, the electronic device 100 may output thedanger warning information 603 on a display included in the vehicle. Forexample, the electronic device 100 may provide text, an image, ananimation, etc. representing the danger warning information 603.

According to an embodiment, the electronic device 100 may determinewhether a position of the vehicle is included in a predetermined range,based on the estimated lane information. Also, the electronic device 100may output guide information based on the determined result. Herein, thepredetermined range may be a position range of the vehicle that isestimated when the vehicle is traveling in a lane.

For example, the electronic device 100 may use a center line of an imageto determine whether the position of the vehicle is included in thepredetermined range. When a location of a lane estimated from a centerline of an acquired image deviates from a safety range, the electronicdevice 100 may determine that a traveling route of the vehicle hasdeparted from the lane. For example, the safety range may have been setin advance in the electronic device 100 based on experiment data.

Also, when the position of the vehicle is not included in thepredetermined range, the electronic device 100 may output guideinformation such as the danger warning information 603.

According to an embodiment, by outputting guide information, theelectronic device 100 may enable a driver to drive the vehicle safelyeven when a visibility obstruction exists in a driving environment.Also, because guide information is output in various forms upon thedriver's driving, the driver's satisfaction in driving may increase.

<FIGS. 7 and 8>

FIGS. 7 and 8 are block diagrams showing a configuration of anelectronic device according to an embodiment.

As shown in FIG. 7, an electronic device 1000 according to an embodimentmay include a memory 1100, a display 1210, a camera 1610, and aprocessor 1300.

However, all of the components shown in FIG. 7 may be not essentialcomponents of the electronic device 1000. The electronic device 1000 maybe configured with more components than those shown in FIG. 7 or withless components than those shown in FIG. 7.

For example, as shown in FIG. 8, the electronic device 1000 according toan embodiment may further include an outputter 1200, a communicator1500, a sensor 1400, an Audio/Video (A/V) inputter 1600, and a userinputter 1700, in addition to the memory 1100, the display 1210, thecamera 1610, and the processor 1300.

The memory 1100 may store programs for processing and control of theprocessor 1300, and also store images input to the electronic device1000 or guide information that is to be output from the electronicdevice 1000. Also, the memory 1100 may store specific information fordetermining whether to output guide information.

The memory 1100 may include at least one type of storage medium among aflash memory type, a hard disk type, a multimedia card micro type, acard type memory (e.g., SD or XD memory), Random Access Memory (RAM),Static Random Access Memory (SRAM), Read Only Memory (ROM), ElectricallyErasable Programmable Read-Only Memory (EEPROM), Programmable Read-OnlyMemory (PROM), a magnetic memory, a magnetic disk, and an optical disk.

The programs stored in the memory 1100 may be classified into aplurality of modules according to their functions, and for example, theprograms may be classified into a User Interface (UI) module 1110, atouch screen module 1120, a notification module 1130, etc.

The UI module 1110 may provide a specialized UI or GUI interworking withthe electronic device 1000 for each application. The touch screen module1120 may sense a user's touch gesture made on a touch screen, andtransfer information about the user's touch gesture to the processor1300. The touch screen module 1120 according to an embodiment mayrecognize a touch code and analyse it. The touch screen module 1120 maybe configured with separate hardware including a controller.

The notification module 1130 may generate a signal for notifying eventoccurrence of the electronic device 1000. Examples of events that occurin the electronic device 1000 may be call signal reception, messagereception, a key signal input, schedule notification, etc. Thenotification module 1130 may output a notification signal in the form ofa video signal through the display 1210, in the form of an audio signalthrough a sound outputter 1220, or in the form of a vibration signalthrough a vibration motor 1230. For example, the notification module1130 may generate a signal for outputting guide information based onestimated lane information.

The outputter 1200 may output an audio signal, a video signal or avibration signal, and the outputter 1200 may include the display 1210,the sound outputter 1220, and the vibration motor 1230.

The display 1210 may display information processed in the electronicdevice 1000. More specifically, the display 1210 may output an imagecaptured by the camera 1610. Also, the display 1210 may synthesize guideinformation created by the processor 1300 with the captured image andoutput the synthesized result.

Also, the display 1210 may display a user interface for executing anoperation related to a response, in response to the user's input.

The sound outputter 1220 may output audio data received from thecommunicator 1500 or stored in the memory 1100. Also, the soundoutputter 1220 may output a sound signal related to a function (e.g.,call signal ringtone, message ringtone, and notification sound) that isperformed in the electronic device 1000. For example, the soundoutputter 1220 may output guide information generated as a signal by thenotification module 1130, as a sound signal, by the control of theprocessor 1300.

The processor 1300 may generally control all operations of theelectronic device 1000. For example, the processor 1300 may execute theprograms stored in the memory 1100 to control all operations of the userinputter 1700, the outputter 1200, the sensor 1400, the communicator1500, the A/V inputter 1600, etc. Also, the processor 1300 may executethe programs stored in the memory 1100 to perform functions of theelectronic device 1000 shown in FIGS. 1 to 6.

More specifically, the processor 1300 may execute one or moreinstructions stored in the memory 1100 to control the camera 1610 tocapture an outside image of the vehicle. Or, the processor 1300 mayexecute one or more instructions stored in the memory 1100 to controlthe communicator 1500 to acquire an outside image of the vehicle.

Also, the processor 1300 may execute one or more instructions stored inthe memory 1100 to determine at least one object for estimating laneinformation from the captured image. For example, the processor 1300 maydetermine a guard rail, a front vehicle, a rear vehicle, etc. includedin the image as at least one object for estimating lane information.

Also, the processor 1300 may execute one or more instructions stored inthe memory 1100, to estimate lane information of a road on which thevehicle is traveling in the image, based on a distance between thedetermined at least one object and the vehicle and a vanishing point ofthe image. For example, the processor 1300 may execute one or moreinstructions stored in the memory 1100 to acquire a distance between thedetermined at least one object and the vehicle through the sensor 1400.Also, the processor 1300 may extract straight lines through a lower endof a building, a guard rail, etc. among objects included in the image tothereby predict a vanishing point.

For example, the processor 1300 may determine a road area based on thedistance between the determined object and the vehicle. Also, theprocessor 1300 may divide the road area by the number of lanes by usingthe vanishing point of the image as a pivot to estimate lane informationof the road.

Or, for example, the processor 1300 may estimate a lane width based onthe distance between the determined object and the vehicle. Also, theprocessor 1300 may extend straight lines having the estimated lane widthby using the vanishing point of the image as a pivot, thereby estimatinglane information of the road.

According to an embodiment, the processor 1300 may execute one or moreinstructions stored in the memory 1100 to determine a road area from thecaptured image. Also, the processor 1300 may acquire information aboutthe number of lanes of the road on which the vehicle is traveling, basedon position information of the vehicle. For example, the processor 1300may execute one or more instructions stored in the memory 1100 tocontrol a position sensor 1460 to acquire GPS information of thevehicle. Also, the processor 1300 may estimate a lane width from theimage, based on the distance between the determined at least one objectand the vehicle, the determined road area, and the acquired informationabout the number of lanes. Also, the processor 1300 may estimate laneinformation of the road on which the vehicle is traveling from theimage, based on the estimated lane width and the vanishing point of theimage.

According to an embodiment, the processor 1300 may execute one or moreinstructions stored in the memory 1100 to determine a traveling centerline of the vehicle. Also, the processor 1300 may execute one or moreinstructions stored in the memory 1100 to determine a first frontvehicle located on the traveling center line as at least one object forestimating lane information. Also, the processor 1300 may execute one ormore instructions stored in the memory 1100 to estimate a first lanewidth of a first lane on which the vehicle is traveling in the imagebased on the distance between the first front vehicle and the vehicleand the vehicle width of the first front vehicle. Also, the processor1300 may execute one or more instructions stored in the memory 1100 toestimate lane information of the road on which the vehicle is travelingin the image, based on the estimated first lane width and the vanishingpoint of the image.

Also, the processor 1300 may determine a second front vehicle that isnot located on the traveling center line, as at least one object forestimating lane information. Also, the processor 1300 may estimate asecond lane width of a second lane on which the second front vehicle istraveling, based on a distance between the second front vehicle and thetraveling center line, a distance between the second front vehicle andthe vehicle, and a vehicle width of the second front vehicle. Also, theprocessor 1300 may estimate lane information of the road on which thevehicle is traveling in the image, based on the estimated second lanewidth and the vanishing point of the image.

According to an embodiment, the processor 1300 may learn a relationshipbetween a plurality of images for the same subject. Also, the processor1300 may execute one or more instructions stored in the memory 1100 tocreate a learning model based on the learned result. Also, the processor1300 may execute one or more instructions stored in the memory 1100 toconvert the captured image using the learning model such that thecaptured image has visibility that is greater than or equal to apredetermined value. Also, the processor 1300 may execute one or moreinstructions stored in the memory 1100 to estimate lane information ofthe road on which the vehicle is traveling by using the converted image.For example, the processor 1300 may determine at least one object forestimating lane information from the converted image.

According to an embodiment, the processor 1300 may execute one or moreinstructions stored in the memory 1100 to control the outputter 1200 tooutput guide information for guiding driving of the vehicle based on theestimated lane information. For example, the processor 1300 may executeone or more instructions stored in the memory 1100 to control thedisplay 1210 to synthesize the guide information with the road area ofthe acquired image and to display the synthesized result. Or, theprocessor 1300 may execute one or more instructions stored in the memory1100 to control the sound outputter 1220 to output the guideinformation. For example, the processor 1300 may execute one or moreinstructions stored in the memory 1100 to determine whether a dangeroccurs on a traveling route of the vehicle based on the guideinformation and predetermined criterion, and to control the soundoutputter 1220 or the vibration motor 1230 to output the determinedresult.

According to an embodiment, the processor 1300 may generate guideinformation. For example, the processor 1300 may create guideinformation based on the estimated lane information and driving speed ofthe vehicle. For example, the processor 1300 may determine whether aposition of the vehicle is included in a predetermined range, based onthe estimated lane information, thereby creating guide information.

The sensor 1400 may sense a state of the electronic device 1000 or astate of surroundings of the electronic device 1000, and transfer thesensed information to the processor 1300.

The sensor 1400 may include at least one among a magnetic sensor 1410,an acceleration sensor 1420, a temperature/humidity sensor 1430, aninfrared sensor 1440, a gyroscope sensor 1450, the position sensor(e.g., GPS) 1460, an atmospheric pressure sensor 1470, a proximitysensor 1480, and a RGB sensor 1490, although not limited thereto.Functions of the individual sensors are intuitively inferred by one ofordinary skill in the art from their names, and therefore, detaileddescriptions thereof will be omitted.

According to an embodiment, the sensor 1400 may measure a distancebetween the at least one object determined from the captured image andthe vehicle.

The communicator 1500 may include one or more components for enablingthe electronic device 1000 to communicate with another device (notshown) and a server (not shown). The other device (not shown) may be acomputing device such as the electronic device 1000 or a sensor,although not limited thereto. For example, the communicator 1500 mayinclude a short-range wireless communication unit 1510, a mobilecommunication unit 1520, and a broadcasting receiver 1530.

The short-range wireless communication unit 1510 may include a Bluetoothcommunication unit, a Bluetooth Low Energy (BLE) communication unit, aNear-Field Communication (NFC) unit, a Wireless Local Access Network(WLAN: Wi-Fi) communication unit, a Zigbee communication unit, anInfrared Data Association (IrDA) communication unit, a Wi-Fi Direct(WFD) communication unit, a Ultra Wideband (UWB) communication module,and an Ant+ communication unit, although not limited thereto. Forexample, the short-range wireless communication unit 1510 may receiveinformation about the number of lanes from a navigation included in thevehicle through short-range wireless communication.

The mobile communication unit 1520 may transmit/receive a wirelesssignal to/from at least one of a base station, an external terminal, anda server on a mobile communication network. Herein, the wireless signalmay include a voice call signal, a video call signal or various formatsof data according to transmission/reception of text/multimedia messages.

The broadcasting receiver 1530 may receive a broadcasting signal and/orbroadcasting-related information from the outside through a broadcastingchannel. The broadcasting channel may include a satellite channel and aterrestrial channel According to implementation examples, the electronicdevice 1000 may not include the broadcasting receiver 1530.

The A/V inputter 1600 may enable a user to input an audio signal or avideo signal, and may include the camera 1610 and the microphone 1620.The camera 1610 may acquire an image frame, such as a still image or amoving image, through an image sensor in a video call mode or aphotographing mode. An image captured by the image sensor may beprocessed by the processor 1300 or a separate image processor (notshown).

According to an embodiment, the camera 1610 may capture an outside imageof the vehicle. For example, the camera 1610 may capture a front imageof the vehicle traveling, although not limited thereto.

The microphone 1620 may receive a sound signal from the outside andprocess the sound signal into electrical voice data. For example, themicrophone 1620 may receive a sound signal from an external device or auser. The microphone 1620 may receive a user's voice input. Themicrophone 1620 may use various noise removal algorithms to remove noisegenerated upon receiving a sound signal from the outside.

The user inputter 1700 may be means to enable the user to input data forcontrolling the electronic device 1000. For example, the user inputter1700 may include a key pad, a dome switch, a touch pad (a capacitivetype, a resistive type, an infrared beam type, a surface acoustic wavetype, an integral strain gauge type, a piezo effect type, etc.), a jogwheel, a jog switch, etc., although not limited thereto.

<FIGS. 9 to 12>

FIG. 9 is a block diagram of a processor according to an embodiment.

Referring to FIG. 9, the processor 1300 according to an embodiment mayinclude a data learner 1310 and a data recognizer 1320.

The data learner 1310 may learn criterion for increasing visibility ofan image. The data learner 1310 may learn criterion related to what datawill be used to increase visibility of an image and how to increasevisibility of an image using data. The data learner 1310 may acquiredata that is to be used for learning, and apply the acquired data to adata determination model which will be described later, thereby learningcriterion for increasing visibility of an image.

The data recognizer 1320 may increase visibility of an input image. Thedata recognizer 1320 may increase visibility of the input image by usingthe learned data determination model. The data recognizer 1320 mayacquire predetermined data according to predetermined criterion obtainedby learning, and apply the data determination model of using theacquired data as an input value to thereby increase visibility of theinput image. Also, a result value output by the data determination modelof using the acquired data as an input value may be used to refine thedata determination model.

At least one of the data learner 1310 and the data recognizer 1320 maybe manufactured in the form of at least one hardware chip and mounted onthe electronic device. For example, at least one of the data learner1310 and the data recognizer 1320 may be manufactured in the form of adedicated hardware chip for artificial intelligence (AI) or as a part ofa typical general-purpose processor (e.g., CPU or application processor)or a graphic dedicated processor (e.g., GPU) and mounted on variouselectronic devices as described above.

In this case, the data learner 1310 and the data recognizer 1320 may bemounted on a single electronic device or on different electronicdevices. For example, one of the data learner 1310 and the datarecognizer 1320 may be included in the electronic device and the otherone may be included in a server. Also, the data learner 1310 may providemodel information constructed by itself to the data recognizer 1320 in awired or wireless manner, and data input to the data recognizer 1320 maybe provided as additional training data to the data learner 1310.

Meanwhile, at least one of the data learner 1310 and the data recognizer1320 may be implemented as a software module. When at least one of thedata learner 1310 and the data recognizer 1320 is implemented as asoftware module (or, a program module including an instruction), thesoftware module may be stored in non-transitory computer readable media.Also, in this case, the at least one software module may be provided byOperating System (OS) or by a predetermined application. Or, a part ofthe at least one software module may be provided by OS and the otherpart may be provided by the predetermined application.

FIG. 10 is a block diagram of a data learner according to an embodiment.

Referring to FIG. 10, the data learner 1310 according to an embodimentmay include a data acquirer 1310-1, a pre-processor 1310-2, a trainingdata selector 1310-3, a model learner 1310-4, and a model evaluator1310-5.

The data acquirer 1310-1 may acquire data required for determining asituation. The data acquirer 1310-1 may acquire data required forlearning to determine a situation.

The data acquirer 1310-1 may receive a plurality of images for the samesubject. For example, the data acquirer 1310-1 may receive imagesobtained by photographing the same road at the same place at differenttimes, in different days, and in different seasons. The data acquirer1310-1 may receive the images through a camera of an electronic deviceincluding the data learner 1310. Or, the data acquirer 1310-1 mayacquire data through an external device capable of communicating withthe electronic device.

The pre-processor 1310-2 may pre-process the acquired data such that theacquired data is used for learning to determine a situation. Thepre-processor 1310-2 may process the acquired data into a predeterminedformat such that the model learner 1310-4 which will be described lateruses the acquired data for learning to determine a situation. Forexample, the pre-processor 1310-2 may divide a plurality of acquiredimages in units of pixels to analyse the plurality of acquired images.Or, the pre-processor 1310-2 may extract an object from each of aplurality of acquired images to analyse the plurality of acquiredimages. Also, the pre-processor 1310-2 may process the extracted objectinto data. Also, the pre-processor 1310-2 may tag objects or pixelslocated at the same location in the images to classify the objects orpixels.

The training data selector 1310-3 may select data required for learningfrom among the pre-processed data. The selected data may be provided tothe model learner 1310-4. The training data selector 1310-3 may selectdata required for learning from among the pre-processed data accordingto predetermined criterion for determining a situation. Also, thetraining data selector 1310-3 may select data according to criterion bylearning of the model learner 1310-4 which will be described later.

The training data selector 1310-3 may select data required for learningfrom among data processed by the pre-processor 1310-2. For example, thetraining data selector 1310-3 may select data corresponding to aspecific object in an image to learn criterion for increasing visibilityof the specific object.

The model learner 1310-4 may learn criterion for a determination of asituation based on training data. Also, the model learner 1310-4 maylearn criterion for what training data will be used to determine asituation.

For example, the model learner 1310-4 may analyse characteristics ofeach object or pixel to learn criterion for increasing visibility of animage. Or, the model learner 1310-4 may analyse a relationship betweenimages by using a pair of pair images to learn criterion for increasingvisibility of the images. For example, the model learner 1310-4 mayextract a difference between images by using a pair of images to therebyanalyse a relationship.

Also, the model learner 1310-4 may learn a data determination model thatis used to determine a situation by using training data. In this case,the data determination model may have been constructed in advance. Forexample, the data determination model may have been constructed inadvance by receiving basic training data (e.g., a sample image, etc.).

The data determination model may be constructed by considering anapplication field of the data determination model, a purpose oflearning, a computing performance of a device, etc. The datadetermination model may be a model based on a neural network. Forexample, a model, such as a DNN, a Recurrent Neural Network (RNN), aBidirectional Recurrent Deep Neural Network (BRDNN), may be used as thedata determination model, although not limited thereto.

According to various embodiments, when there are a plurality of datadetermination models constructed in advance, the model learner 1310-4may determine a data determination model having a high relevance betweeninput training data and basic training data as a data determinationmodel that is to be learned. In this case, the basic training data mayhave been classified in advance according to data types, and the datadetermination models may have been classified in advance according todata types. For example, the basic training data may have beenclassified in advance according to various criteria, such as regionswhere the training data has been created, times at which the trainingdata has been created, sizes of the training data, genres of thetraining data, creators of the training data, kinds of objects in thetraining data, etc.

Also, the model learner 1310-4 may learn the data determination model byusing a learning algorithm including error back-propagation or gradientdescent.

Also, the model learner 1310-4 may learn the data determination modelthrough, for example, supervised learning using training data as aninput value. Also, the model learner 1310-4 may learn the datadetermination model through, for example, unsupervised learning thatfinds criterion for determining a situation by itself learning a kind ofdata required for determining a situation without any supervision. Also,the model learner 1310-4 may learn the data determination model through,for example, reinforcement learning using a feedback informing whetherthe result of a situation determination according to learning iscorrect.

Also, after the data determination model is learned, the model learner1310-4 may store the learned data determination model. In this case, themodel learner 1310-4 may store the learned data determination model in amemory of an electronic device including the data recognizer 1320. Or,the model learner 1310-4 may store the learned data determination modelin a memory of an electronic device including the data recognizer 1320which will be described later. Or, the model learner 1310-4 may storethe learned data determination model in a memory of a server connectedto the electronic device through a wired/wireless network.

In this case, the memory in which the learned data determination modelis stored may store, for example, a command or data related to at leastanother component of the electronic device, together. Also, the memorymay store software and/or a program. The program may include, forexample, a kernel, middleware, an application programming interface(API), and/or an application program (or “application”), etc.

The model evaluator 1310-5 may input evaluation data to the datadetermination model, and when the result of recognition output from theevaluation data fails to satisfy predetermined criterion, the modelevaluator 1310-5 may instruct the model learner 1310-4 to performlearning again. Herein, the evaluation data may be predetermined datafor evaluating the data determination model.

For example, the evaluation data may include at least one pair of pairimages.

For example, when the model evaluator 1310-5 determines that a number orportion of evaluation data showing incorrect results of recognitionamong the results of recognition of the learned data determination modelwith respect to the evaluation data exceeds a predetermined thresholdvalue, the model evaluator 1310-5 may determine that the learned datadetermination model has failed to satisfy the predetermined criterion.For example, it is assumed that the predetermined criterion is definedas 2%. In this case, when the learned data determination model outputswrong recognition results with respect to more evaluation data than 20evaluation data among a total of 1000 evaluation data, the modelevaluator 1310-5 may determine that the learned data determination modelis improper.

Meanwhile, when there are a plurality of learned data determinationmodels, the model evaluator 1310-5 may determine whether each learneddata determination model satisfies the predetermined criterion, anddetermine a leaned data determination model satisfying the predeterminedcriterion as a final data determination model. In this case, when aplurality of learned data determination models satisfy the predeterminedcriterion, the model evaluator 1310-5 may determine a predeterminedlearned data determination model or a predetermined number of learneddata determination models among the plurality of learned datadetermination models in evaluation score order as a final datadetermination model.

Meanwhile, at least one of the data acquirer 1310-1, the pre-processor1310-2, the training data selector 1310-3, the model learner 1310-4, andthe model evaluator 1310-5 in the data learner 1310 may be manufacturedin the form of at least one hardware chip and mounted on the electronicdevice. For example, at least one of the data acquirer 1310-1, thepre-processor 1310-2, the training data selector 1310-3, the modellearner 1310-4, and the model evaluator 1310-5 in the data learner 1310may be manufactured in the form of a dedicated hardware chip for AI oras a part of a typical general-purpose processor (e.g., CPU orapplication processor) or a graphic dedicated processor (e.g., GPU) andmounted on various electronic devices as described above.

Also, the data acquirer 1310-1, the pre-processor 1310-2, the trainingdata selector 1310-3, the model learner 1310-4, and the model evaluator1310-5 may be mounted on a single electronic device or on differentelectronic devices. For example, a part of the data acquirer 1310-1, thepre-processor 1310-2, the training data selector 1310-3, the modellearner 1310-4, and the model evaluator 1310-5 may be included in theelectronic device, and the other part may be included in a server.

Also, at least one of the data acquirer 1310-1, the pre-processor1310-2, the training data selector 1310-3, the model learner 1310-4, andthe model evaluator 1310-5 may be implemented as a software module. Whenat least one of the data acquirer 1310-1, the pre-processor 1310-2, thetraining data selector 1310-3, the model learner 1310-4, and the modelevaluator 1310-5 is implemented as a software module (or, a programmodule including an instruction), the software module may be stored innon-transitory computer readable media. Also, in this case, the at leastone software module may be provided by OS or by a predeterminedapplication. Or, a part of the at least one software module may beprovided by OS and the other part may be provided by the predeterminedapplication.

FIG. 11 is a block diagram of the data recognizer 1320 according to anembodiment.

Referring to FIG. 11, the data recognizer 1320 according to anembodiment may include a data acquirer 1320-1, a pre-processor 1320-2, arecognition data selector 1320-3, a recognition result provider 1320-4,and a model refiner 1320-5.

The data acquirer 1320-1 may acquire data required for determining asituation, and the pre-processor 1320-2 may pre-process the acquireddata such that the data acquired for determining the situation is used.The pre-processor 1320-2 may process the acquired data into apredetermined format such that the recognition result provider 1320-4which will be described later uses the data acquired for determining thesituation.

The recognition data selector 1320-3 may select data required fordetermining a situation from among the pre-processed data. The selecteddata may be provided to the recognition result provider 1320-4. Therecognition data selector 1320-3 may select the entire or a part of thepre-processed data according to predetermined criterion for determininga situation. Also, the recognition data selector 1320-3 may select dataaccording to criterion determined in advance by learning of the modellearner 1310-4 which will be described later.

The recognition result provider 1320-4 may apply the selected data to adata determination model to determine a situation. The recognitionresult provider 1320-4 may provide the result of recognition accordingto a purpose of recognition of data. The recognition result provider1320-4 may use data selected by the recognition data selector 1320-3 asan input value to apply the selected data to the data determinationmodel. Also, the result of recognition may be determined by the datadetermination model.

For example, the result of recognition of an input image may be providedas text, an image, or an instruction (e.g., an application executioninstruction, a module function execution instruction, etc.). Therecognition result provider 1320-4 may apply an image to the datadetermination model to convert the image into an image satisfying apredetermined visibility reference value, and provide the convertedimage.

For example, the recognition result provider 1320-4 may provide adisplay function execution instruction as the result of recognition tocause the display 1210 to output the converted image.

The model refiner 1320-5 may refine the data determination model basedon an evaluation on the result of recognition provided by therecognition result provider 1320-4. For example, the model refiner1320-5 may provide the result of recognition provided by the recognitionresult provider 1320-4 to the model learner 1310-4 to cause the modellearner 1310-4 to refine the data determination model.

Meanwhile, at least one of the data acquirer 1320-1, the pre-processor1320-2, the recognition data selector 1320-3, the recognition resultprovider 1320-4, and the model refiner 1320-5 in the data recognizer1320 may be manufactured in the form of at least one hardware chip andmounted on the electronic device. For example, at least one of the dataacquirer 1320-1, the pre-processor 1320-2, the recognition data selector1320-3, the recognition result provider 1320-4, and the model refiner1320-5 may be manufactured in the form of a dedicated hardware chip forAI or as a part of a typical general-purpose processor (e.g., CPU orapplication processor) or a graphic dedicated processor (e.g., GPU) andmounted on various electronic devices as described above.

Also, the data acquirer 1320-1, the pre-processor 1320-2, therecognition data selector 1320-3, the recognition result provider1320-4, and the model refiner 1320-5 may be mounted on a singleelectronic device or on different electronic devices. For example, apart of the data acquirer 1320-1, the pre-processor 1320-2, therecognition data selector 1320-3, the recognition result provider1320-4, and the model refiner 1320-5 may be included in the electronicdevice and the other part may be included in a server.

Also, at least one of the data acquirer 1320-1, the pre-processor1320-2, the recognition data selector 1320-3, the recognition resultprovider 1320-4, and the model refiner 1320-5 may be implemented as asoftware module. When at least one of the data acquirer 1320-1, thepre-processor 1320-2, the recognition data selector 1320-3, therecognition result provider 1320-4, and the model refiner 1320-5 isimplemented as a software module (or a program module including aninstruction), the software module may be stored in non-transitorycomputer readable media. Also, in this case, the at least one softwaremodule may be provided by OS or by a predetermined application. Or, apart of the at least one software module may be provided by OS and theother part may be provided by the predetermined application.

FIG. 12 is a diagram for describing an example in which the electronicdevice 1000 according to an embodiment interworks with a server 2000 tolearn and recognize data.

Referring to FIG. 12, the server 2000 may learn criterion for increasingvisibility of an image, and the electronic device 1000 may determine asituation based on the result of learning by the server 2000.

In this case, a model learner 2340 of a data learner 2300 of the server2000 may perform a function of the data learner 1310 shown in FIG. 10.The data learner 2300 includes a data acquirer 2310, a pre-processor2320, a training data selector 2330, a model learner 2340, and a modelevaluator 2350, which may correspond to the data acquirer 1310-1, thepre-processor 1310-2, the training data selector 1310-3, the modellearner 1310-4, and the model evaluator 1310-5, respectively, of datalearner 1310, and a full description thereof will not be repeated. Themodel learner 2340 of the server 2000 may learn criterion related towhat data will be used to increase visibility of an image and how toincrease visibility of an image using data. The model learner 2340 mayacquire data that is to be used for learning, and apply the acquireddata to a data determination model which will be described later,thereby learning criterion for increasing visibility of an image.

Also, the recognition result provider 1320-4 of the electronic device1000 may apply data selected by the recognition data selector 1320-3 toa data determination model created by the server 2000 to increasevisibility of an image. For example, the recognition result provider1320-4 may transmit data selected by the recognition data selector1320-3 to the server 2000, and request the server 2000 to apply the dataselected by the recognition data selector 1320-3 to a determinationmodel to increase visibility of the image. Also, the recognition resultprovider 1320-4 may receive an image whose visibility has been increasedby the server 2000 from the server 2000.

Or, the recognition result provider 1320-4 of the electronic device 1000may receive a determination model created by the server 2000 from theserver 2000, and increase visibility of the image by using the receiveddetermination model. In this case, the recognition result provider1320-4 of the electronic device 1000 may apply the data selected by therecognition data selector 1320-3 to a data determination model receivedfrom the server 2000 to increase visibility of the image.

An embodiment may be implemented in the form of a computer-readablerecording medium including an instruction that is executable by acomputer, such as a program module that is executed by a computer. Thecomputer-readable recording medium may be an arbitrary available mediumwhich is able to be accessed by a computer, and may include a volatileor non-volatile medium and a separable or non-separable medium. Further,the computer-readable recording medium may include a computer storagemedium and a communication medium. The computer storage medium mayinclude volatile and non-volatile media and separable and non-separablemedia implemented using an arbitrary method or technology for storinginformation such as a computer readable instruction, a data structure, aprogram module, or other data. The communication medium may generallyinclude a computer readable instruction, a data structure, a programmodule, other data of a modulated data signal such as a carrier wave, oranother transmission mechanism, and include an arbitrary informationtransmission medium.

Also, in the present specification, the term “unit” may be a hardwarecomponent such as a processor or a circuit, and/or a software componentthat is executed by a hardware component such as a processor.

It should be understood that the above descriptions of the presentdisclosure are merely for illustrative purposes, and therefore, it willbe apparent that those skilled in the art can readily make variousmodifications thereto without changing the technical spirit andessential features of the present disclosure. Thus, it should beunderstood that the embodiments described above are merely forillustrative purposes and not for limitation purposes in all aspects.For example, each component described as a single type may beimplemented in a distributed form, and likewise, components described ina distributed form may be implemented in a combined form.

The scope of the present disclosure is shown by the claims to bedescribed below rather than the detailed description, and it is to beconstrued that the meaning and scope of the claims and all modificationsor modified forms derived from the equivalent concept thereof areencompassed within the scope of the present disclosure.

The invention claimed is:
 1. An electronic device comprising: a cameraconfigured to capture an outside image of a vehicle; a memory storingone or more instructions; and a processor configured to execute the oneor more instructions stored in the memory, wherein the processorexecutes the one or more instructions to: determine, from the capturedimage, at least one object for estimating lane information, wherein thelane information comprises lines that divide a road on which the vehicleis driving along lanes, determine a road area from the captured image,acquire information about a number and determined different ratios oflanes of the road on which the vehicle is driving, based on informationof the road at a location corresponding to position information of thevehicle, estimate a lane width from the captured image, based on adistance between the determined at least one object and the vehicle, thedetermined road area, a vanishing point of the image, and the acquiredinformation about the number of lanes, divide the road area by theacquired number of lanes by using the vanishing point of the image as apivot and the estimated lane width to estimate the lane information ofthe road, and output guide information for guiding driving of thevehicle based on the estimated lane information, wherein the laneinformation further comprises a first lane width of a first lane onwhich the vehicle is traveling and a lane position of the vehicle in thefirst lane, wherein the guide information comprises a warning of adanger of lane departure, wherein the information about the number anddetermined different ratios of lanes of the road on which the vehicle isdriving is acquired based on at least one of global positioning system(GPS) position information of the vehicle, a local navigation includedin the vehicle, or information stored in advance in the electronicdevice, and wherein different respective widths of the lanes of the roadare estimated based on the respective determined ratios of the lanes. 2.The electronic device of claim 1, wherein the processor further executesthe one or more instructions to: acquire an artificial intelligencelearning model based on a result obtained by using a plurality of imagesfor a same subject to learn a relationship between the plurality ofimages for the same subject; convert the captured image by using theacquired artificial intelligence learning model such that the capturedimage has visibility that is greater than or equal to a predeterminedvalue; and determine the at least one object for estimating the laneinformation from the converted image.
 3. The electronic device of claim1, wherein the determined at least one object for estimating laneinformation comprises at least one among a guard rail, another vehicledriving ahead, or another vehicle driving behind.
 4. The electronicdevice of claim 1, wherein the processor further executes the one ormore instructions to: determine a driving center line of the vehicle;estimate, when the determined at least one object includes a first frontvehicle located on the driving center line, the first lane width of thefirst lane on which the vehicle is driving from the image, based on adistance between the first front vehicle and the vehicle and a vehiclewidth of the first front vehicle; and estimate the lane information fromthe image based on the estimated first lane width, the vanishing pointof the image, and the determined driving center line of the vehicle. 5.The electronic device of claim 4, wherein the processor further executesthe one or more instructions to: estimate, when the determined at leastone object includes a second front vehicle not located on the drivingcenter line, a second lane width of a second lane on which the secondfront vehicle is driving, based on a distance between the second frontvehicle and the driving center line, a distance between the second frontvehicle and the vehicle, and a vehicle width of the second frontvehicle; and estimate the lane information based on the estimated secondlane width, the vanishing point of the image, and a driving center lineof the second front vehicle.
 6. The electronic device of claim 1,further comprising a display displaying the captured image, wherein theprocessor further executes the one or more instructions to: create theguide information based on the estimated lane information and a drivingspeed of the vehicle, and synthesize the created guide information withthe road area of the captured image to display a result of thesynthesizing on the display.
 7. The electronic device of claim 1,wherein the processor further executes the one or more instructions to:determine whether the position of the vehicle in the first lane isincluded in a predetermined range, based on the estimated laneinformation; and output the guide information based on a result of thedetermining of the lane position of the vehicle in the first lane. 8.The electronic device of claim 1, wherein the guide information isoutput in a form of displaying at least one of a text, an image, or ananimation.
 9. A method comprising: acquiring an outside image of avehicle; determining, from the acquired image, at least one object forestimating lane information, wherein the lane information compriseslines that divide a road on which the vehicle is driving along lanes;determining a road area from the acquired image; acquiring informationabout a number and determined different ratios of lanes of the road onwhich the vehicle is driving, based on information of the road at alocation corresponding to position information of the vehicle;estimating a lane width from the acquired image, based on a distancebetween the determined at least one object and the vehicle, thedetermined road area, a vanishing point of the image, and the acquiredinformation about the number of lanes; dividing the road area by theacquired number of lanes by using the vanishing point of the image as apivot and the estimated lane width to estimate the lane information ofthe road; and outputting guide information for guiding driving of thevehicle based on the estimated lane information, wherein the laneinformation further comprises a first lane width of a first lane onwhich the vehicle is traveling and a lane position of the vehicle in thefirst lane, wherein the guide information comprises a warning of adanger of lane departure, and wherein the information about the numberand determined different ratios of lanes of the road on which thevehicle is driving is acquired based on at least one of globalpositioning system (GPS) position information of the vehicle, a localnavigation included in the vehicle, or information stored in advance,and wherein different respective widths of the lanes of the road areestimated based on the respective determined ratios of the lanes. 10.The method of claim 9, further comprising: learning a relationship amonga plurality of images for a same subject by using the plurality ofimages for the same subject; and converting the acquired image based ona result of the learning such that the acquired image has visibilitythat is greater than or equal to a predetermined value, wherein thedetermining of the at least one object for estimating the laneinformation from the acquired image comprises: determining the at leastone object for estimating the lane information from the converted image.11. The method of claim 9, further comprising: determining a drivingcenter line of the vehicle, wherein when the determined at least oneobject includes a first front vehicle located on the driving centerline, and wherein the estimating of the lane information of the road onwhich the vehicle is driving from the image based on the distancebetween the determined at least one object and the vehicle and thevanishing point of the image comprises: estimating, from the image, thefirst lane width of the first lane on which the vehicle is driving,based on the distance between the first front vehicle and the vehicleand a vehicle width of the first front vehicle, and estimating the laneinformation from the image, based on the estimated first lane width, thevanishing point of the image, and the determined driving center line ofthe vehicle.
 12. The method of claim 11, wherein when the determined atleast one object includes a second front vehicle not located on thedriving center line, the estimating of the lane information from theimage, based on the distance between the determined at least one objectand the vehicle and the vanishing point of the image comprises:estimating, from the image, a second lane width of a second lane onwhich the second front vehicle is driving, based on a distance betweenthe second front vehicle and the driving center line, a distance betweenthe second front vehicle and the vehicle, and a vehicle width of thesecond front vehicle; and estimating the lane information from the imagebased on the estimated second lane width, the vanishing point of theimage, and a driving center line of the second front vehicle.
 13. Themethod of claim 9, wherein the outputting of the guide information forguiding driving of the vehicle, based on the estimated lane information,comprises: creating the guide information based on the estimated laneinformation and a driving speed of the vehicle; and synthesizing thecreated guide information with the road area of the acquired image todisplay a result of the synthesizing.
 14. The method of claim 9, whereinthe outputting of the guide information for guiding driving of thevehicle, based on the estimated lane information, comprises: determiningwhether the position of the vehicle in the first lane is included in apredetermined range, based on the estimated lane information; andoutputting the guide information based on a result of the determining ofthe lane position of the vehicle in the first lane.
 15. Acomputer-readable recording medium storing a program for executing themethod of claim 9, on a computer.